Structure and Nonrigid Motion Analysis from 2d Images

Structure and nonrigid motion analysis is a very important problem in computer vision. Most of the work in nonrigid motion analysis thus far has aimed at correspondence estimation from the complete structure of the scene (stereo or range), \optic ow" analysis of image sequences, or 2D image registrations. Limited study has been conducted to track both structure and nonrigid motion at the same time from time-varying 2D images. This problem is quite challenging not only due to the absence of correspondence but also due to the unavoidable structure ambiguity of the input 2D images. In this thesis, a general scheme is proposed to perform structure and nonrigid motion analysis from 2D image sequences without correspondences. Within this scheme, two modules are developed: The local analysis module tracks local structure and nonrigid motion within small segmented areas in order to recover small nonrigid details and the global analysis module constrains possible nonrigid behaviors by using appropriate global constraints. An iterative algorithm is designed to integrate these two modules together into a global-local framework. Two branches of applications (non-shape-based applications and shape-based applications) are discussed with detailed examples in this thesis. For the cloud motion analysis which is a non-shape-based application, a set of non-shape-based global constraints is de ned according to the physical uid dynamics. Moreover, motion-compensated cloud image interpolation is performed for the purpose of cloud visualization by using the tracked 3D cloud motion correspondences. For the human facial motion analysis which is a shape-based application, a new geometric shape